Anagn rish ProtÔpwn & Neurwnikˆ DÐktua Probl mata 2

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Anagn rish ProtÔpwn & Neurwnikˆ DÐktua Probl mata 2"

Αγαθίας Μακρής
5 χρόνια πριν
Προβολές:

1 Jeìdwroc Alexìpouloc, Anaplhrwt c Kajhght c Theodoros Alexopoulos, Associate Professor EJNIKO METSOBIO POLUTEQNEIO NATIONAL TECHNICAL UNIVERSITY SQOLH EFARMOSMENWN MAJHMATIKWN KAI DEPARTMENT OF PHYSICS FUSIKWN EPISTHMWN - TOMEAS FUSIKHS ZOGRAFOU CAMPUS HRWWN POLUTEQNEIOU ATHENS - GREECE AJHNA Phone : , Fax: Thl: , Fax: Theodoros.Alexopoulos@cern.ch theoalex@central.ntua.gr html:// Anagn rish ProtÔpwn & Neurwnikˆ DÐktua Probl mata (Epistrof : IanouarÐou 004) 0. Upˆrqoun diˆforoi trìpoi gia na genikeôsoume thn idèa twn sunart sewn diˆkrishc dôo klˆsewn se C klˆseic ω, ω,..., ω k. 'Enac trìpoc ja eðnai na qrhsimopoi soume (C ) sunart seic diˆkrishc, ètsi ste an g k ( x ) > 0 tìte to deðgma x ω k, en an g k ( x ) < 0 tìte x ω k. Me th bo jeia enìc paradeðgmatoc se dôo diastˆseic gia C = 3 (treic klˆseic), na deðxete ìti autìc o trìpoc taxinìmhshc mporeð na odhg sei se perioqèc sto deigmatoq ro twn x gia tic opoðec h taxinìmhsh ja eðnai asaf c. 'Enac ˆlloc trìpoc eðnai na qrhsimopoi soume mia sunˆrthsh diˆkrishc g jk ( x ) gia kˆje dunatì zeôgoc klˆsewn ω j kai ω k, ètsi ste gia g jk ( x ) > 0 to x ω j kai gia g jk ( x ) < 0 to x ω k. Gia C klˆseic, apaitoôntai C(C )/ sunart seic diˆkrishc. Me èna aplì parˆdeigma se dôo diastˆseic gia C = 3, na deðxete ìti kai autìc o trìpoc ja odhg sei se asafeðc perioqèc taxinìmhshc. Gia treic klˆseic C = 3, ja upˆrqoun dôo grammikèc sunart seic diˆkrishc g ( x )g ( x ). ja isqôei: gia x C, tìte g ( x ) > 0 kai gia x C, tìte g ( x ) > 0. Autì ja odhg sei sto akìloujo prìblhma. P c ja mporèsoume na taxinom soume deðgmata x ta opoða èqoun thn idiìthta: g ( x ) > 0 KAI g ( x ) > 0. Profan c tètoia x ja an koun kai stic dôo klˆseic, dhlad C KAI C. To sq ma perigrˆfei autì to prìblhma. Akìmh kai sthn perðptwsh pou jewr soume tic dôo sunart seic diˆkrishc parˆllhlec metaxô

2 g ( x ) > 0 g ( x ) < 0 g ( x ) < 0 g ( x ) > 0 Sq ma : Oi dôo grammikèc sunart seic diˆkrishc profan c orðzoun perioq tou q rou (grammoskiasmènh perioq ) pou taxinomeðtai stic dôo klˆseic sugqrìnwc. touc, de ja èqoume lôsh tou probl matoc, kajìti h tom twn g ( x ) > 0 KAI g ( x ) > 0 eðnai èna mh-kenì sônolo. ParathreÐste oti h tom twn g ( x ) > 0 KAI g ( x ) > 0 eðnai èna kenì sônolo mìno sthn perðptwsh pou oi dôo grammikèc sunart seic diˆkrishc sumpðptoun, pou shmaðnei g ( x ) = g ( x ). Gia treic klˆseic C = 3, ja upˆrqoun C(C )/ = 3 sunart seic diˆkrishc, g ( x ), g 3 ( x ), g 3 )( x ). O kanìnac taxinìmhshc ja eðnai wc akoloôjwc:. An g ( x ) > 0 KAI y 3 ( x ) > 0, tìte x C.. An g ( x ) < 0 KAI y 3 ( x ) > 0, tìte x C. 3. An g 3 ( x ) < 0 KAI y 3 ( x ) < 0, tìte x C 3. Autìc o kanìnac taxinìmhshc odhgeð sto akìloujo prìblhma, ìpwc faðnetai sto sq ma 6. perioq de mporeð na taxinomhjeð. H akìloujh. g ( x ) < 0 KAI g 3 ( x ) > 0.. g ( x ) > 0 KAI g 3 ( x ) > 0 KAI g 3 ( x ) < 'Estw ta 6 deðgmata ekpaðdeushc pou an koun se dôo klˆseic ω, ω, ìpwc faðnontai sto sq ma (3):

3 g ( x ) < 0 g ( x ) > 0 C 3 C g 3 ( x ) > 0 g 3 ( x ) < 0 g 3 ( x ) < 0 C g 3 ( x ) > 0 Sq ma : Oi treic sunart seic diˆkrishc profan c orðzoun perioq tou q rou pou den taxinomeðtai. [ Z Z [ Sq ma 3: 3

4 ( ) ( ) ( ) ( ) ( ) ( ) 0 S =,,,,,. 0 0 }{{}}{{} ω Na orðsete èna neur na (perceptron) pou na orðsei thn taxinìmhsh aut n twn dôo klˆsewn. Dhlad, orðste tic sunˆyeic w, w kai to kat fli w 0. ω [ Z Z [ Z \ Z L L L [ Z Sq ma 4: x επιφάνεια απόφασης x - Sq ma 5: 4

5 Profan c h epifˆneia apìfashc eðnai: x = x + x x + = 0, ìpwc faðnetai sto sq ma 5. Epomènwc h exðswsh aut mporeð na jewrhjeð wc : w x + w x + w 0 = 0, () ìpou w =, w =, kai w 0 =, kai h exðswsh () eðnai: w = ( w w ) = ( / ), w 0 =, w t x + w0 = 0, me x = ( x x ) t. x w Σ x w w 0 συνάρτηση ενεργοποίησης Sq ma 6: 'Ara h taxinìmhsh èqei wc akoloôjwc: 5

6 { ( + ) w t x + w0 = < 0 ω : ( ) w t x + w 0 = / < 0 ω : ( 0 0 ) w t x + w 0 = > 0 ( ) w t x + w 0 = > 0 ( 0 ) w t x + w 0 = > 0 ( ) w t x + w 0 = > 0 }. 0.3 JewreÐste ton algìrijmo thc apìtomhc pt shc (steepest descent) ìpou h sunˆrthsh kìstouc J(w) = k (w w 0 ) + k. (a) BreÐte th bèltisth lôsh w pou elaqistopoieð th sunˆrthsh kìstouc J(w). (b) Me th bo jeia tou algorðjmou thc apìtomhc pt shc: w(i + ) = w(i) + ϱ dj dw, na breðte mia analutik èkfrash tou w(i). JumhjeÐte p c epilôame diaforoexis seic apì thn Anˆlush S matoc! Na breðte th sunj kh pou prèpei na upakoôei h parˆmetroc tou rujmoô ekmˆjhshc ϱ, ste o algìrijmoc thc apìtomhc pt shc na sugklðnei. (g) Na breðte to ìrio tou w(i) gia i. (a) (b) O algìrijmoc thc apìtomhc pt shc ja eðnai: dj dw = k (w w 0 ) = 0 w = w 0. w(i + ) = w(i) + ϱk (w(i) w 0 ) w(i + ) = ( ϱk )w(i) + ϱk w 0 i w(i) = ( ϱk ) i w(0) + ( ϱk ) j ϱk w 0 6 j=0

7 w(i) = ( ϱk ) i ϱk w 0 w(0) + ( ϱk ) Epomènwc to w i gia na sugklðnei, ja prèpei na isqôei: w(i) = ( ϱk ) i w(0) + w 0. () ϱk < < ϱk < (g) Apì th sqèsh (): k < ϱ < 0 lim w(i) = w(0). i 0.4 'Estw x eðnai mia tuqaða metablht me mèsh tim µ kai pðnaka diasporˆc Σ. Na deðxete ìti o Σ eðnai jetikˆ orismènoc, dhlad na deðxete ìti: 'Estw y èna mh mhdenikì diˆnusma. Tìte: y t Σ y > 0 y 0. y t Σ y = y t E [ ( x µ )( x µ ) t] y = E [ y t ( x µ )( x µ ) t y ]. (3) OrÐzoume mia nèa bajmwt metablht : A = y t ( x µ ). (4) ParathroÔme oti to A eðnai mia tuqaða metablht me mèsh tim mhdèn. Apì tic sqèseic (3) kai (4) ja èqoume: y t Σ y = E[A ] = σa > 0, ìpou σ A eðnai h diasporˆ thc tuqaðac metablht c A. 7

8 0.5 'Estw to diˆnusma x = (x x ) t eðnai katanemhmèno katˆ Gauss me puknìthtec pijanìthtac p( x /ω i ) N( µ i, σ I). Na sqediˆsete tic epifˆneiec apìfashc gia èna taxinomht elˆqisthc eukleðdeiac apìstashc an èqoume 5 klˆseic me tic akìloujec mèsec timèc: µ = (0 0) t, µ = ( 0) t, µ 3 = ( 0) t, µ 4 = ( ) t, µ 5 = ( ) t. Oi epifˆneiec apìfashc gia tic pènte klˆseic faðnontai sto sq ma 7. x x - Sq ma 7: 8

9 0.6 DÐnontai ta parakˆtw dianôsmata ekpaðdeushc: ( ) ( ) ( ) ( ) ( ) 0, 0, 0, 5,, S =,,,,, 0, 0, 0, 0, 8, }{{} ( ) ( ) ( ) ( ) ( ),, 5 0, 9 0, 0,,,,,. 0, 0, 5 0,, 0, 9 }{{} ω Na deðxete grafikˆ ìti autˆ den eðnai grammikˆ diaqwrðsima gegonìta, kai na sqediasteð mia katˆllhlh arqitektonik Perceptron pou na ta diaqwrðzei. ω x,5 ω ω 0,5 - + g ( x ) = ,5,5 g ( x ) = 0 x Sq ma 8: Apì to sq ma 8, profan c oi dôo klˆseic den eðnai grammikˆ diaqwrðsimec. MporoÔme na dialèxoume dôo eujeðec: x + x = 0, kai x + x 3 3 = 0, ste ta deðgmata thc klˆshc ω na brejoôn metaxô twn dôo epifanei n diˆkrishc. Ja qreiastoôme dôo str mata gia th sqedðash tou Perceptron. To krufì str ma ja perièqei dôo neur nec me exìdouc y, y. O q roc y, y faðnetai sto sq ma 9. 9

10 x x y y 0, -0, 0 0 0, 0, 0 0-0,5 0, 0 0, 0,8,,, -0, 0,5 0,6 0 0,9 0, 0 0,, 0 0, 0,9 0 y ω ω 0,5 ω y Sq ma 9: Dhlad ja mporèsoume na èqoume mia epifˆneia apìfashc: y y = 0, pou mporeð na diaqwrðsei ta ω, ω, ìpwc faðnetai sto sq ma.epomènwc to neurwnikì dðktuo ja eðnai autì pou faðnetai sto sq ma Upojèste ìti dôo sônola S kai S sto q ro R l eðnai grammikˆ diaqwrðsima. Dhlad, upˆrqei èna diˆnusma w R l kai èna bajmwtì w 0 R, ètsi ste: w t x + w0 = { > 0 x S < 0 x S 0

11 x y x - y / y Sq ma 0: JewreÐste ta nèa dianôsmata wc akoloôjwc: ( x ) x =, w = ( w w 0 ). (a) Na deðxete ìti w tˆx > 0 ˆx, an to prìshmo twn dianusmˆtwn ˆx tou sunìlou S allˆzei. (b) Na perigrˆyete grafikˆ tð sumbaðnei sto er thma (a) ìtan l =, S = { 3,, }, kai S = {5, 6, 7}. (a) Profan c isqôei: w t x w0 = ( w w 0 ) ( x ) = w tˆx. An x S w tˆx > 0. An x S w tˆx < 0 w t ( )ˆx > 0. Epomènwc an to prìshmo tou ˆx allˆzei ìtan to x S, ja isqôei h sqèsh w tˆx > 0 ˆx. (b) H allag tou pros mou perigrˆfetai sto sq ma. Metˆ apì thn allag thc metablht c se ˆx kai thn allag tou pros mou ìla ta deðgmata brðskontai sthn Ðdia pleurˆ thc epifˆneiac apìfashc.

12 x x x x πριν µετά Sq ma : 0.8 Ston algìrijmo tou neur na (Perceptron) èqoume: w (i + ) = w (i) + ϱ(i) x i, ìtan to x i èqei taxinomhjeð lanjasmèna, dhlad, w t x i < 0. Na breðte th sunj kh pou dièpei thn parˆmetro tou rujmoô ekmˆjhshc ϱ(i), ètsi ste to x na taxinomhjeð orjˆ sto b ma i +. An w (i) t x i < 0, tìte to ˆx i èqei taxinomhjeð lanjasmèna sto i ostì b ma tou algorðjmou. Epomènwc gia na èqoume th swst taxinìmhsh tou x i sto epìmeno b ma, to w (i) ja prèpei na allˆxei me tètoio trìpo ste w (i + ) t x i > 0. O algìrijmoc eðnai: w (i + ) = w (i) + ρ x i w (i + ) t x i = ( w (i) + ρ x i ) t x i ω (i + ) x i = w t x i + ρ x i > 0 ρ > w t x i x i.

13 0.9 'Estw ìti ta sônola S, S eðnai grammikˆ diaqwrðsima, kai èqoume gegonìta ekpaðdeushc x = S, x = S kai x 3 = S. H arqik tim tou dianôsmatoc w eðnai w (0) = (, ) t, kai h parˆmetroc tou rujmoô ekmˆjhshc eðnai ϱ = 0, 5. Na prosdioristeð to diˆnusma w ste o algìrijmoc tou Perceptron na mac d sei thn orj taxinìmhsh. Ja qrhsimopoi soume ton algìrijmo tou Perceptron. Se kˆje b ma tou algorðjmou, ja kanonikopoioôme ta dianôsmata bˆrouc ŵ = w / w. H orjìthta thc taxinìmhshc ja elègqetai gia kˆje b ma. Ta dianôsmata bˆrouc ja dðnontai apì th sqèsh: ŵ(i + ) = ŵ(i) 0, 5 3 σ xi x i, ìpou σ xi = an to deðgma x i ω allˆ taxinomeðtai sthn klˆsh ω, σ xi = + an to deðgma x i ω allˆ taxinomeðtai sthn klˆsh ω, kai se ìlec tic peript seic σ xi = 0. B ma Diˆnusma Kanonikopoihmèno DeÐgmata algorðjmou bˆrouc diˆnusma bˆrouc me lˆjoc taxinìmhsh 0 w = ( ) t (0, 707 0, 707) t x, x 3 w = (, 9 0, 7) t ( 0, 88 0, 48) t x w = ( 0, 38 0, 0) t (, 00 0, 05) t x 3 w = ( 0, 50 0, 55) t ( 0, 67 0, 74) t kanèna i= 0.0 (a) Na efarmìsete ton aplì algìrijmo eôreshc twn protôpwn gia ta dianôsmata: {( ) ( ) ( ) ( ) ( ) ( )} S =,,,,, JewreÐste ìti to kat fli T = 3. (b) Na efarmìsete th mèjodo MaxMin sto deigmatoq ro S tou erwt matoc (a). (g) Na efarmìsete th mèjodo twn K-mèswn (K-means) sto deigmatoq ro S tou erwt matoc (a). (a) EÐnai parìmoia thc ˆskhshc?? (b) blèpe ˆskhsh?? (g) blèpe ˆskhsh?? 3

Σχετικά έγγραφα

1 η Σειρά Ασκήσεων Θεόδωρος Αλεξόπουλος. Αναγνώριση Προτύπων και Νευρωνικά Δίκτυα

1 η Σειρά Ασκήσεων Θεόδωρος Αλεξόπουλος. Αναγνώριση Προτύπων και Νευρωνικά Δίκτυα Σχολή Εφαρμοσμένων Μαθηματικών και Φυσικών Επιστημών Εθνικό Μετσόβιο Πολυτεχνείο Αναγνώριση Προτύπων και Νευρωνικά Δίκτυα η Σειρά Ασκήσεων Θεόδωρος Αλεξόπουλος Άδεια Χρήσης Το παρόν εκπαιδευτικό υλικό